Skip to content

Conversation

@edenreich
Copy link
Contributor

@edenreich edenreich commented May 30, 2025

Summary

This Pull Request (PR) add an example for MCP (Model Context Protocol) using the Inference Gateway SDK. It includes a sample client that demonstrates how to use the MCP features, such as tool invocation and context management. The example is designed to help users understand how to integrate MCP into their applications effectively.

Changelog

  • docs(examples): Add example environment configuration and README files for SDK usage
  • docs: Update README to include pre-requisites for using the Typescript SDK examples
  • docs(examples): Update example links in README and add list example README
  • docs(examples): Add examples for chat, list, and MCP using Inference Gateway SDK
  • docs(exmaples): Add sample files and configuration for MCP example
  • docs(examples): Update LLM model references to remove provider prefix
  • docs(mcp): Add note to ensure no containers are running before starting the example
  • feat: Support both local tools and remote MCP tools; Add onMCPTool callback
  • docs(examples): Refactor MCP examples and client for improved error handling and token tracking

edenreich added 9 commits May 27, 2025 18:31
…s for SDK usage

Signed-off-by: Eden Reich <eden.reich@gmail.com>
…t SDK examples

Signed-off-by: Eden Reich <eden.reich@gmail.com>
…EADME

Signed-off-by: Eden Reich <eden.reich@gmail.com>
…Gateway SDK

- Created a chat example with package.json, tsconfig.json, and index.ts to demonstrate chat completions and streaming responses.
- Added a list example with package.json, tsconfig.json, and index.ts for listing models and MCP tools.
- Included README.md files for both chat and list examples with instructions for getting started.
- Added .gitignore files to exclude node_modules in both list and MCP examples.

Signed-off-by: Eden Reich <eden.reich@gmail.com>
- Created a new directory with sample files for the MCP filesystem server, including `sample.txt`, `data.json`, and `config.yaml`.
- Added a README.md file to describe the purpose and usage of the sample data.
- Implemented a TypeScript configuration file (`tsconfig.json`) for the MCP example.
- Created a package.json and package-lock.json for managing dependencies in the examples directory.
- Updated the OpenAPI specification to include new endpoints and improved formatting for consistency.

Signed-off-by: Eden Reich <eden.reich@gmail.com>
Signed-off-by: Eden Reich <eden.reich@gmail.com>
…ng the example

Signed-off-by: Eden Reich <eden.reich@gmail.com>
…llback

- Created a new MCP Web Search Server with a package.json file.
- Updated the main MCP package.json to include example scripts for remote and local tools.
- Added a .gitignore file in the shared directory to exclude unnecessary files.
- Removed outdated README.md, config.yaml, data.json, and sample.txt files from the shared directory.
- Introduced a new sample_sales_data.csv file for demonstration purposes.
- Enhanced the client.ts to support MCP tool callbacks and improved streaming response processing.

Signed-off-by: Eden Reich <eden.reich@gmail.com>
…andling and token tracking

- Updated docker-compose.yml to set environment to production and commented out optional Ollama service.
- Enhanced example-mcp-tools.ts to use console.info for logging and added model availability checks.
- Modified index.ts to implement token tracking for prompt and completion tokens, displaying usage metrics at the end.
- Improved filesystem MCP server (index.js) to use console.info for logging and ensure consistent messaging.
- Updated web-search MCP server (index-http.js and index.js) to use console.info for logging and improved request handling.
- Enhanced InferenceGatewayClient to handle mid-stream errors and added type definitions for stream chunks that may contain errors.

Signed-off-by: Eden Reich <eden.reich@gmail.com>
@edenreich edenreich merged commit d771356 into main May 30, 2025
1 check passed
@edenreich edenreich deleted the docs/add-more-examples-how-to-use-this-sdk branch May 30, 2025 00:15
ig-semantic-release-bot bot added a commit that referenced this pull request May 30, 2025
## [0.7.2](v0.7.1...v0.7.2) (2025-05-30)

### 📚 Documentation

* Add more examples how to use this SDK ([#15](#15)) ([d771356](d771356))

### 🔧 Miscellaneous

* Add Docker-in-Docker feature to development container ([177e9f3](177e9f3))
* Remove MCP documentation references and update related instructions ([b33c08f](b33c08f))
@ig-semantic-release-bot
Copy link
Contributor

🎉 This PR is included in version 0.7.2 🎉

The release is available on:

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants